Video survey of pre-grasp interactions in natural hand activities

Objects are often movable in the environment and do not have to be grasped from the presented placement. •Pre-grasp interaction can adjust object configuration in the environment to improve the task conditions for the final grasp. •Our video observation surveys the variety of pre-grasp interactions used by people in natural task settings. •The observed pre-grasp interactions can be described by the object re-configuration and the underlying intent to improve the posture or grasp quality. Related work Recent robotics research work has investigated the interdependency between the components of a manipulation action. •Berenson et al. (2007) and Gienger et al. (2008) investigated ways to plan the grasp choice on an object in coordination with the planning of the arm motion. • Stilman and Kuffner (2005) presented a planner for Navigation Among Movable Obstacles, where clutter must be moved to find a clear navigation path. In this work, we focus on the relation between pre-grasp interaction and the final object grasp. Surveyed activities Category Session (number of participants, total footage recorded) Food preparation Potluck lunch set up (5 people, 24 min.) Cooking dinner (1 person, 21 min.) Cooking at a restaurant (4 people, 30 min.) Housekeeping / cleaning Party set up (5 people, 11 min.) Lab clean up (14 people, 13 min.) Fish tank maintenance (1 person, 5 min.) Office work Desk work (1 person, 5 min.) Library work (1 person, 10 min.) Other Bicycle repair (3 people, 34 min.) Moving furniture (3 people, 12 min.) Example pre-grasp interactions Bimanual re-grasping tumbled a pan before the final grasp by the left hand. A scooping action reshaped a pile of peelings before the final lift. Books were slid off the top of a stack to grasp the bottom surface. A bucket handle was separated from the bucket body to achieve a hook-grasp. The sheet of paper was curled at the corner to reach the bottom surface for a pinch grasp. A cabinet was tumbled to reach a handhold for a whole body grasp. Taxonomy of object reconfiguration aspect One aspect of pre-grasp interaction is the object reconfiguration before the final grasp. The observed examples were classified based on the degrees of freedom which were adjusted by the pre-grasp interaction. 3-DoF planar displacement Rigid transform Non-rigid reconfiguration 6-DoF tumbling Articulation Continuous deformation 1-DoF rotation Set arrangement Object adjustment mug books cabinet bucket paper peelings stapler CD cases pan ladder rug bolts pan pitcher wrench foil paper stack large box paperclip plastic bag set of jars plates spatula chopped produce food carton chopsticks books Underlying intent of pre-grasp interaction The second aspect of a pre-grasp interaction is how it improves the task conditions for the final grasp compared to the initial presented conditions. Postural preferences After object adjustment, the whole body posture for grasping may load the joints less or have improved stance stability. This may be especially relevant for grasping heavy objects in demanding tasks. Natural grasp with rotation Direct grasp without rotation Grasp preferences Pre-grasp interaction can also expose new object surfaces which are contacted in the final desired grasp. This was frequently observed for stacked objects or cluttered environments. Other potential quality metrics • inertia of the object with respect to gravity • perceived time for re-using a practiced motion versus planning a new grasp Implications for robotic manipulation •Grasping is part of an entire manipulation movement which includes any pre-grasp interaction and the subsequent post-grasp task. •Reach-to-grasp actions should be planned with consideration of the object’s movability in the environment instead of assuming fixed object placement. •Pre-grasp interaction which adjusts the object could improve the posture and grasp quality in a manipulation action. •A library of pre-grasp interactions could be added to a robot’s repertoire for versatile manipulation planning of complex tasks. Robotics Science and Systems 2009 University of Washington, Seattle email: {lillianc,nsp}@cs.cmu.edu 28 June 2009 Workshop: Understanding the Human Hand for Advancing Robotic Manipulation